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Abstract 


A new hardware approach to implementation of image processing algorithms is described. 
The approach is based on silicon devices which would permit an independent analog processing 
channel to be dedicated to every pixel. A laminar architecture consisting of a stack of planar 
arrays of the devices would form a two-dimensional array processor with a 2-D array of inputs 
located directly behind a focal plane detector array. A 2-D image data stream would propa- 
gate in neuronlike asynchronous pulse coded form through the laminar processor. Such systems 
would integrate image acquisition and image processing. Acquisition and processing would be 
performed concurrently as in natural vision systems. The research is aimed at implementation 
of algorithms, such as the intensity dependent summation algorithm and pyramid processing 
structures, which are motivated by the operation of natural vision systems. Implementation 
of natural vision algorithms would benefit from the use of neuronlike information coding and 
the laminar, 2-D parallel, vision system type architecture. Besides providing a neural network 
framework for implementation of natural vision algorithms, a 2-D parallel approach could elimi- 
nate the serial bottleneck of conventional processing systems. Conversion to serial format would 
occur only after raw intensity data has been substantially processed. An interesting challenge 
arises from the fact that the mathematical formulation of natural vision algorithms does not 
specify the means of implementation, so that hardware implementation poses intriguing ques- 
tions involving vision science. 
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1 Introduction 


Spontaneous generation of neuronlike action potential pulses in voltage or current driven silicon 
p + -n-n + diodes at liquid helium temperatures has been studied extensively.[l,2,3,4,5,6] A simple 
circuit used to generate these pulses (Fig. 1) consists of a p + -n-n + diode and a load resistor, 
capacitances and a current source. 

In Fig. 2, we show how an optical sensor can be embedded in the circuit of Fig. 1. Such 
a circuit permits single stage coding of optical information into neuronlike spiketrains. The 
simplicity of the coding circuit would permit fully parallel, asynchronous processing of a two 
dimensional array of signals as would emerge from a 2-D array of photodetectors, i.e. a focal 
plane array. See Figs. 3 and 4. 
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Figure 1: A circuit used to generate spontaneous neuronlike pulses. 

The only active circuit element is a p + -n-n + diode. 


Parallel asynchronous spiketrain signal processing would occur as in neural networks. The 
recent upsurge of interest in neural networks is an encouraging sign that the means of processing 
discussed here may be closely connected with significant new trends in signal processing and 
information processing. 

By fully parallel processing, we mean one processing channel per pixel. This point is easily 
appreciated when one considers possible NASA image processing applications involving arrays 
of 1000 by 1000 pixels at 1 kilohertz frame rates. A fully parallel approach requires kilohertz 
processing in each channel while a fully serial approach would require processor speeds on the 
order of gigahertz. Processed output data may be much more condensed than raw input intensity 
data, so that conversion to a serial data stream after parallel processing is a very good strategy 
for many applications. 


1.1 Hardware Implementation of Image Processing Algorithms 

The above observations strongly suggest that our approach would be especially advantageous as 
a means of implementation of image processing schemes which are biologically motivated. An 
example of such an approach is given in the work of Marr and Hildreth[7] on edge detection and 
related general discussion of the computational viewpoint is given in Marr’s influential book on 
vision. [8] 
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Figure 2: DC coupling of the current from a photodetector 
to a circuit which generates neuronlike output pulses. In the 
case of infrared illumination, a silicon impurity band detec- 
tor is typically employed while in the case of near infrared, 
visible or ultraviolet illumination a reverse biased silicon p-i-n 
photodiode is employed. 
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2.6 Hardware Implementation of the IDS Image Processing Algo- 
rithm 

Our approach to circuit design and hardware implementation is guided in part by the fact that 
the IDS algorithm was intended to be in accord with several key features of natural vision 
systems. Thus, the algorithm or a close approximation to the algorithm is being implemented 
by biological neural networks whose general structure is known. See Figs. 7 and 8. On the 
other hand, no direct link between the IDS algorithm and retinal neural network architecture 
yet exists. Understanding this link would be of direct significance to circuit designs for parallel 
asynchronous neuronlike implementation of the IDS algorithm and, in addition, would be of 
significance in the field of vision. 

A hardware implementation of the IDS algorithm need not have an exact retinal neural 
network analog. Therefore, a clear-cut conceptual advance in relation to retinal implementation, 
although desirable, is not a necessary condition for IDS hardware implementation. However, 
even without detailed understanding of retinal processing, circuit design efforts can benefit from 
knowledge of the general features of retinal neural network architecture, such as those apparent 
in Figs. 7 and 8. 

To illustrate our approach, we show in Fig. 10 a preliminary strategy for implementation of 
the IDS algorithm which possesses some architectural similarity to retinas. 

A key feature of the implementation concept is a 2-D array of constant current sources, in 
one-to-one correspondence with the photodetectors. The lateral spreading of this current is 
associated with the IDS point-spread function. Pulses associated with the spiketrain coding 
of the photodetector outputs gate the forward flow of current from the current sources. High 
intensities provide more rapid gating and more forward current flow which competes with and 
limits lateral spreading of the current. Thus, higher intensities diminish lateral spreading as 
in the IDS algorithm. On the other hand, the constancy of each current source and current 
conservation during spreading produce a constraint that the integrated output current must also 
be constant, despite its intensity dependent spreading. This is analogous to the IDS constant 
“volume” constraint, i.e. constant intensity x area. [9] 

A key aspect of the circuit involves capacitive couplings as in Fig. 9 which permit information 
transfer, but no net time-averaged current flow, i.e. no dc component. This permits light 
intensity to play a role but the photocurrent does not add to the dc current coming from the 
constant current sources. 

The output is again coded into spiketrains for further processing or for output typically via 
an LED array. Note that retinal outputs to the brain are coded into spiketrains by the ganglion 
cells. 

The implementation concepts described here are preliminary concepts. It is very likely 
that further considerations will be needed to produce quantitative agreement with the intensity 
dependence of IDS spatial scaling associated with the point-spread function from the input point 
(x,y) to the output point (p,q) 

J(x,y) x S [l[x,y) x ((x - p) 2 + (y - y) 2 )] (1) 

where the non-negative real function S is normalized by: 

/ oo r co 

/ s(p 2 + q 2 )dpdq = 1 (2) 
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Figure 10: Functional description of a pre- 
liminary circuit concept for implementation 
of the IDS algorithm. 


Pixel or detector size and the spatial sampling period (or frequency) are important issues 
in connection with fundamental image processing . This idea has been emphasized in the 

work of Huck, Fales, McCormick, Park, Halyo, Samms and Stacy. [20, 21, 22] The issue is not 
dealt with in the continuum formulation of the IDS algorithm. [9] 

Furthermore, there is an issue with respect to circuit architecture which is similar to an issue 
raised by Cornsweet 2 in connection with retinal implementation. This concerns implementing 
IDS point-spread functions (one for each input) on a shared network structure. For linear point- 
spread functions, this is easy to envisage. However, with the nonlinear intensity dependence of 
the IDS algorithm, one worries that nonlinear spreading associated with one photoreceptor will 
interfere with the spreading associated with another photoreceptor if spreading occurs over a 
shared network. Non-shared spreading networks would solve this problem but would be more 
complex (higher parts count) and would contradict the impression that retinal neural networks 
(as shown in Figs. 7 and 8) are shared. This issue deserves further study. 
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